Real-time modelling and forecasting
How can we better prepare
for the next pandemic?
Sebastian Funk (@sbfnk)
https://epiforecasts.io
9 September, 2022
ETH Zurich
Acknowledgements
EpiForecasts group (https://epiforecasts.io)
Akira Endo, Hannah Choi, Hugo Gruson, James Munday,
Kath Sherratt, Nikos Bosse, Sam Abbott, Sophie Meakin
Johannes Bracher, Nick Reich and other collaborators
Mechanistic models support causal understanding, but predictions can have value in their own right
Real-time forecasts can inform decision making
Anticipate healthcare demand
Real-time forecasts can inform decision making
Inform interventions
Real-time forecasts can inform decision making
Support clinical trials
What we did
Forecasting via the renewal equation
\begin{align}
\textrm{New infections}~I(t) & = R_t \sum_{\tau} g_{\tau} I_{t-\tau}\\
\textrm{Reproduction number}~R(t) & = R_{t-1} \exp(\textrm{GP})\\
\textrm{Delayed reporting}~D(t) & = \sum_\tau \xi_\tau I_{t-\tau} \\
\textrm{Observations}~C(t) & = \mathrm{NegBin}(D_t \omega_{(t \textrm{mod} 7)}, \phi)
\end{align}
Global COVID case forecasts via the renewal equation
Forecasting to inform policy in the UK
COVID forecast hubs
- How good were COVID forecasts?
- How could COVID forecasts be improved?
- What can we conclude for the next pandemic?
How good were COVID forecasts?
We can compare forecasts using
proper scoring rules, e.g.
\[\mathrm{CRPS}(F, x) = \mathbb{E}|X-x| - \frac{1}{2}\mathbb{E}|X-X'|\]
Forecast hubs:
median ensemble outperforms individual models
We can compare forecasts using proper scoring rules, e.g. \[\mathrm{CRPS}(F, x) = \mathbb{E}|X-x| - \frac{1}{2}\mathbb{E}|X-X'|\] but these only tell us about relative quality of forecasts
Absolute quality of forecasts #1: baseline models
Absolute quality of forecasts #2: human vs. machine
Absolute quality of forecasts #3: calibration
How could COVID forecasts be improved?
Ensembles: learn from past performance
Including possible predictors #1: leading surveillance
Including possible predictors #2: observed behaviour
Munday et al., in prep
Including possible predictors #3: variants
What can we conclude for the next pandemic?
Summary
- Outbreak forecasting has very limited in predictive ability
- Ensembles perform best, but can be difficult to interpret
Open questions:
- Can predictive performance be improved?
- Are we measuring predictive performance in the right way?
Forecasting and nowcasting will remain relevant
Nowcasting of delayed reports
New initiatives
We need collaborative efforts, using standardised datasets to compare methods and generating sustainable tools
“We were losing ourselves in details […] all we needed to know is, are the number of cases rising, falling or levelling off?”
Hans Rosling, Liberia, 2014